CLOSE: Curriculum Learning on the Sharing Extent Towards Better One-Shot NAS

One-shot Neural Architecture Search (NAS) has been widely used to discover architectures due its efficiency. However, previous studies reveal that one-shot performance estimations of might not be well correlated with their performances in stand-alone training because the excessive sharing operation parameters (i.e., large extent) between architectures. Thus, recent methods construct even more over-parameterized supernets reduce extent. But these improved introduce a number extra and thus cause an undesirable trade-off costs ranking quality. To alleviate above issues, we propose apply Curriculum Learning On Sharing Extent (CLOSE) train supernet both efficiently effectively. Specifically, extent (an easier curriculum) at beginning gradually decrease (a harder curriculum). support this strategy, design novel (CLOSENet) decouples from operations realize flexible scheme adjustable Extensive experiments demonstrate CLOSE can obtain better quality across different computational budget constraints than other supernets, is able superior when combined various search strategies. Code available https://github.com/walkerning/aw_nas .